Identifying stale entries in address translation cache

ABSTRACT

A mapping may be changed in a table stored in memory. The table may map a first set of addresses, for a set of data, to a second set of addresses. The changing of the mapping may including mapping the first set of addresses to a third set of addresses. In response to the changing of the mapping, one or more flush operations may be executed to invalidate one or more entries within one or more address translation caches. The one or more entries may include the second set of addresses. In response to the executing of the one or more flush operations, a first test case may be run. The first test case may be to test whether any of the first set of addresses are mapping to the second set of addresses.

BACKGROUND

This disclosure relates generally to virtual memory systems, and morespecifically, to identifying one or more stale entries in an addresstranslation cache within such systems.

Processors may operate in physical or virtual mode. In physical mode,any requested memory address may directly correlate with a physicaladdress within actual system memory (i.e., main memory) without addresstranslation. In virtual mode, any requested memory address may betranslated from a virtual address (including an effective address) to aphysical address. In virtual mode, when an address is requested, theoperating system may determine if the virtual address is located withina page table of the main memory. The page table may store variousmappings of virtual addresses to physical addresses as page tableentries (PTEs). Accordingly, if the virtual address is located withinthe page table, a physical address may be obtained.

Because page table lookup may be expensive, a translation cache (e.g.,Translation Lookaside Buffer (TLB)) may help data access become moreefficient. For every data request, a first set of accesses to mainmemory may have to occur for data fetching and a second set of accessesto main memory may have to occur for address translation within a pagetable. An address translation cache, such as a TLB, may help reduce dataaccess cost by bringing translation lookup closer to the processor,thereby reducing translation latency. A TLB is a specialized type ofcache within the processor that retains recently-accessed page addresstranslations (i.e., virtual to physical address translations). For agiven data request, a search for the data may begin in the processor atthe TLB. When a corresponding entry is not found within the TLB, theoperating system may then search the main memory within the page tablefor the corresponding entry.

SUMMARY

One or more embodiments are directed to a computer-implemented method, asystem, and/or a computer program product for identifying stale entrieswithin an address translation cache. A mapping may be changed in a tablestored in memory. The table may map a first set of addresses, for a setof data, to a second set of addresses. The changing of the mapping mayincluding mapping the first set of addresses to a third set ofaddresses. In response to the changing of the mapping, one or more flushoperations may be executed to invalidate one or more entries within oneor more address translation caches. The one or more entries may includethe second set of addresses. In response to the executing of the one ormore flush operations, a first test case may be run. The first test casemay be to test whether any of the first set of addresses are mapping tothe second set of addresses.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example computing device,according to embodiments.

FIG. 2 is an example diagram illustrating how addresses are translatedin a virtual memory system, according to embodiments.

FIG. 3 is a flow diagram of an example process for identifying staleentries within address translation cache, according to embodiments.

FIG. 4A is a diagram of an example page table entry illustrating how aphysical address may be changed, according to embodiments.

FIG. 4B is a diagram of an example segment table entry illustrating howa virtual address may be changed, according to embodiments.

FIG. 5 is a diagram of an example of two test cases, according toembodiments.

While the invention is amenable to various modifications and alternativeforms, specifics thereof have been shown by way of example in thedrawings and will be described in detail. It should be understood,however, that the intention is not to limit the invention to theparticular embodiments described. On the contrary, the intention is tocover all modifications, equivalents, and alternatives falling withinthe spirit and scope of the invention.

DETAILED DESCRIPTION

Aspects of the present disclosure relate to virtual memory systems, moreparticular aspects relate to identifying one or more stale entries in anaddress translation cache within such systems. While the presentdisclosure is not necessarily limited to such applications, variousaspects of the disclosure may be appreciated through a discussion ofvarious examples using this context.

An operating system and/or processor (e.g., a memory management unit(MMU)) may issue one or more flush operations to invalidate (remove)stale entries from an address translation cache. Flush operations arealso known as “kill” (e.g., TLB kill), “invalidate,” or “shootdown”(e.g., TLB shootdown) operations. Flushing or invalidating stale entriesmay occur by setting all valid bits to 0 to indicate that the entriesare no longer valid for address translation. A “stale” entry asdescribed herein is an entry within address translation cache (e.g.,TLB) that is no longer valid for address translation (e.g.,effective-to-physical address translation). Accordingly, the stale entrymay correspond to an invalid or old address mapping.

A context switch may lead to address translation caches containing staleentries. A context switch as disclosed herein may be an address space(e.g., virtual or effective) that switches or changes address mappingsbecause of a changed process or logical partition. Consequently, whenswitching from one process or logical partition to another, the hardwareand/or operating system should ensure that the about-to-be-run processdoes not accidently utilize address translations within addresstranslation cache from some previously run process or logical partition.For example, when one process (P1) is running, it may assume the TLBmight be caching translations that are valid for it (i.e., translationsthat come from P1's page table). The 10th virtual page of P1 may bemapped to physical frame 100. If another process (P2) exists, theoperating system may perform a context switch and consequently, the 10thvirtual page of P2 may be mapped to physical frame 170. However, the10^(th) virtual page may map to two different physical frames (100 and170), because of a stale entry within the TLB (physical frame 100), butthe hardware may not be able to distinguish which entry is stale and maytherefore map to the wrong physical frame. One method to deal with thisis to simply flush out all or some of the entries in the TLB. In anotherexample, a hypervisor may perform a context switch between logicalpartitions.

One or more stale entries may be flushed out of a TLB for various otherreasons other than a context switch. For example, when a virtual page ispaged out to disk, the PTE in the page table may be marked as notpresent. If that page still has a TLB entry, it may be marked as stale(assuming that the TLB includes only present PTEs). In another example,a process may map a file into memory, access a few pages in the mappedarea, and then unmap the file. Even after the unmapping, the TLB maystill include entries that were inserted when the mapped area wasaccessed, but because the mapping no longer exists, those entries may bestale.

In a multiprocessor system (e.g., Symmetric Multiprocessing System(SMP)), each processor may include its own shared address translationcache and whenever there is a change in address mapping due to, forexample a context switch, each processor (or processor core) may have tosynchronize its TLB and page table with the other TLBs and page tablesto reflect the change. For example, when a mapping change happens at afirst address associated with a first process, a first processor(performing the first process) may perform the change, flag a flushrequest at a shared memory location, and then trigger an Inter-ProcessorInterrupt (IPI) to other processors. When those processors receive theinterrupt, they may flush one or more associated entries within theirown TLB cache, and acknowledge that the request has been completed. Thefirst processor may have to wait until all processors acknowledge thecompletion.

Each processor (or processor core) may issue these flush operationssimultaneously or substantially simultaneous. However, when flushoperations are performed simultaneously, they may become lost in theinterconnect fabric, thereby causing stale entries to remain within theaddress translation cache. In addition, there may be a significant timedifference for various cores of various processors to receive andacknowledge the flush operation because of proximity of the core thatissued a flush operation with other cores that need to receive the flushoperation message. For example, in a system that includes 256 cores,core 0 may issue a first TLB flush operation for a first address and thecommunication may take longer to reach core 256 than core 1. However,core 256 may have issued its own second TLB flush operation for a secondaddress at substantially the same time as it receives an instructionfrom core 1 to invalidate the first address based on the first TLBflush. However, because the first TLB flush operation may be processedat substantially the same time as the second TLB flush operation andbecause there may be no priority scheme (or other locking mechanism) inplace, the TLB entry corresponding to the first address may not beinvalidated from core 256's TLB. This example illustrates a racecondition. A race condition may occur when a system attempts to performoperations at the same time or substantially the same time, but theoperations may be performed in an unintended order or not performed atall, thereby causing a system bug (e.g., maintaining a stale entrywithin a TLB). Accordingly, embodiments of the present disclosure aredirected to identifying stale entries within an address translationcache in an efficient manner.

FIG. 1 is a block diagram illustrating an example computing device,according to embodiments. The components of the computing device 100 caninclude one or more processors 106 and 132, a memory system 112, aterminal interface 118, a storage interface 120, an Input/Output (“I/O”)device interface 122, and a network interface 124, all of which arecommunicatively coupled, directly or indirectly, for inter-componentcommunication via a memory bus 110, a processor bus 134, an I/O bus 116,bus interface unit (“IF”) 108, and an I/O bus interface unit 114.

The computing device 100 may include one or more general-purposeprogrammable central processing units (CPUs) 106A, 106B, 132A, and 132B,herein generically referred to as the processors 106 and 132. In someembodiments, each of the CPUs (106A, 106B, 132A, and 132B) may beindividual processor cores. In an embodiment, the computing device 100may contain multiple processors; however, in another embodiment, thecomputing device 100 may alternatively be a single CPU device. Eachprocessor executes instructions stored in the memory system 112.

In some embodiments, each processor may include a memory management unit(MMU) that includes a TLB and a Segment Lookaside Buffer (SLB), both ofwhich may be address translation caches. For example, processor 106 mayinclude TLB 136 and SLB 138 and processor 132 may include TLB 140 andSLB 142. The SLBs may be utilized for mapping effective addresses tovirtual addresses and the TLBs may be utilized for mapping virtualaddresses to physical addresses such that address translation may occur,as described in more detail below.

The computing device 100 may include a bus interface unit 108 to handlecommunications among the processors 106 and 132, the memory system 112,the display system 104, and the I/O bus interface unit 114. The I/O businterface unit 114 may be coupled with the I/O bus 116 for transferringdata to and from the various I/O units. The I/O bus interface unit 114may communicate with multiple I/O interface units 118, 120, 122, and124, which are also known as I/O processors (IOPs) or I/O adapters(IOAs), through the I/O bus 116. The display system 104 may include adisplay controller, a display memory, or both. The display controllermay provide video, audio, or both types of data to a display device 102.The display memory may be a dedicated memory for buffering video data.The display system 104 may be coupled with a display device 102, such asa standalone display screen, computer monitor, television, a tablet orhandheld device display, or another other displayable device. In anembodiment, the display device 102 may include one or more speakers forrendering audio. Alternatively, one or more speakers for rendering audiomay be coupled with an I/O interface unit. In alternate embodiments, oneor more functions provided by the display system 104 may be on board anintegrated circuit that also includes the processor 106 and/or 132. Inaddition, one or more of the functions provided by the bus interfaceunit 108 may be on board an integrated circuit that also includes theprocessor 106 and/or 132.

The I/O interface units support communication with a variety of storageand I/O devices. For example, the terminal interface unit 118 supportsthe attachment of one or more user I/O devices, which may include useroutput devices (such as a video display devices, speaker, and/ortelevision set) and user input devices (such as a keyboard, mouse,keypad, touchpad, trackball, buttons, light pen, or other pointingdevices). A user may manipulate the user input devices using a userinterface, in order to provide input data and commands to the user I/Odevice 126 and the computing device 100, may receive output data via theuser output devices. For example, a user interface may be presented viathe user I/O device 126, such as displayed on a display device, playedvia a speaker, or printed via a printer.

The storage interface 120 supports the attachment of one or more diskdrives or direct access storage devices 128 (which are typicallyrotating magnetic disk drive storage devices, although they couldalternatively be other storage devices, including arrays of disk drivesconfigured to appear as a single large storage device to a hostcomputer, or solid-state drives, such as a flash memory). In anotherembodiment, the storage device 128 may be implemented via any type ofsecondary storage device. The contents of data within the memory system112, or any portion thereof, may be stored to and retrieved from thestorage device 128 as needed. The I/O device interface 122 provides aninterface to any of various other I/O devices or devices of other types,such as printers or fax machines. The network interface 124 provides oneor more communication paths from the computing device 100 to otherdigital devices and computer systems.

Although the computing device 100 shown in FIG. 1 illustrates aparticular bus structure providing a direct communication path among theprocessors 106, the memory system 112, the bus interface 108, thedisplay system 104, and the I/O bus interface unit 114, in alternativeembodiments the computing device 100 may include different buses orcommunication paths, which may be arranged in any of various forms, suchas point-to-point links in hierarchical, star or web configurations,multiple hierarchical buses, parallel and redundant paths, or any otherappropriate type of configuration. Furthermore, while the I/O businterface unit 114 and the I/O bus 108 are shown as single respectiveunits, the computing device 100, may include multiple I/O bus interfaceunits 114 and/or multiple I/O buses 116. While multiple I/O interfaceunits are shown, which separate the I/O bus 116 from variouscommunication paths running to the various I/O devices, in otherembodiments, some or all of the I/O devices are connected directly toone or more system I/O buses.

In various embodiments, the computing device 100 is a multi-usermainframe computer system, a single-user system, or a server computer orsimilar device that has little or no direct user interface, but receivesrequests from other computer systems (clients). In other embodiments,the computing device 100 may be implemented as a desktop computer,portable computer, laptop or notebook computer, tablet computer, pocketcomputer, telephone, smart phone, or any other suitable type ofelectronic device. In some embodiments, the computing device 100 is aSymmetric Multiprocessing System (SMP). An SMP may be a system thatincludes two or more processors (e.g., processors 106 and 132) thatshare a memory and an operating system instance. An SMP's processors mayperform multiple processes simultaneously.

In an embodiment, the memory system 112 may include a random-accesssemiconductor memory, storage device, or storage medium (either volatileor non-volatile) for storing or encoding data and programs. In anotherembodiment, the memory system 112 represents the entire virtual memoryof the computing device 100, and may also include the virtual memory ofother computer systems coupled to the computing device 100 or connectedvia a network 130. The memory may be a single monolithic entity, but inother embodiments the memory system 112 may include a hierarchy ofcaches and other memory devices. For example, memory may exist inmultiple levels of caches, and these caches may be further divided byfunction, so that one cache holds instructions while another holdsnon-instruction data, which is used by the processor. Memory system 112may be further distributed and associated with different CPUs or sets ofCPUs, as is known in any various so-called non-uniform memory access(NUMA) computer architectures.

The memory system 112 may store all or a portion of the components anddata shown (e.g., test case module 146, page table 148, and segmenttable 150). These programs and data structures are illustrated in FIG. 1as being included within the memory system 112 in the computing device100; however, in other embodiments, some or all of them may be ondifferent computer systems and may be accessed remotely, e.g., via anetwork 130. The computing device 100 may use virtual addressingmechanisms that allow the programs of the computing device 100 to behaveas if they only have access to a large, single storage entity instead ofaccess to multiple, smaller storage entities. Thus, while the componentsand data shown in FIG. 1 are illustrated as being included within thememory 112, these components and data are not necessarily all completelycontained in the same storage device at the same time. Although thecomponents and data shown in FIG. 1 are illustrated as being separateentities, in other embodiments some of them, portions of some of them,or all of them may be packaged together.

In some embodiments, the memory system 112 may include a test casemodule 146, a page table 148, and a segment table 150. The test casemodule 146 may be a module that identifies whether there are any staleentries within an address translation cache, as described in more detailbelow. In some embodiments, the test case module 146 is a part of thecomputing device 100 operating system. A page table (also known as apage map) is a data table structure stored in the memory system 112 thatis utilized by a virtual memory system to store mappings of thelow-order bits of virtual addresses to the low-order bits of physicaladdresses. A segment table is a data table structure stored in thememory system 112 that is utilized by a virtual memory system to storemappings from the high-order bits effective addresses to the high-orderbits of virtual addresses.

In an embodiment, the components and data shown in FIG. 1 may includeinstructions or statements that execute on the processors 106 and/or 132or instructions or statements that are interpreted by instructions orstatements that execute the processors 106 and/or 132 to carry out thefunctions as further described below. In another embodiment, thecomponents shown in FIG. 1 may be implemented in hardware viasemiconductor devices, chips, logical gates, circuits, circuit cards,and/or other physical hardware devices in lieu of, or in addition to, aprocessor-based system. In an embodiment, the components shown in FIG. 2may include data in addition to instructions or statements.

FIG. 1 is intended to depict representative components of the computingdevice 100. Individual components, however, may have greater complexitythan represented in FIG. 1. In FIG. 1, components other than or inaddition to those shown may be present, and the number, type, andconfiguration of such components may vary. Several particular examplesof additional complexity or additional variations are disclosed herein;these are by way of example only and are not necessarily the only suchvariations. The various program components illustrated in FIG. 1 may beimplemented, in various embodiments, in a number of different ways,including using various computer applications, routines, components,programs, objects, modules, data structures etc., which may be referredto herein as “software,” “computer programs,” or simply “programs.”

FIG. 2 is an example diagram illustrating how addresses are translatedin a virtual memory system, according to embodiments. FIG. 2 includes aneffective address 202 (that comprises an Effective Segment ID (ESID), apage index, and byte offset), an SLB 208A, a segment table 208B, avirtual address 204 (that comprises a virtual segment ID (VSID), pageindex, and byte offset), a TLB 210A, a page table 210B, and a physicaladdress 206 (that includes a physical page number (also referred toherein as a real page number or RPN) and a byte offset).

In some embodiments, address translation in FIG. 2 may start when arequest for data within memory is made. A program may first referencememory using the effective (logical) address 202 (e.g., 64 bits)computed by a processor when it executes a load, store, branch or cacheinstruction, etc. Each process may process may include its own uniqueeffective address space. This allows every process to run as though itis the only process running on the system and has all required memoryresources available. In order to translate the effective address 202 tothe virtual address 204, the operating system or other component mayperform a table lookup within the SLB 208A and, if needed, the segmenttable 208B. The SLB 208A and/or segment table 208B within main memorymay include at least two fields—an ESID field and VSID field.Accordingly, the operating system or other component may translate theeffective address 202 to the virtual address 204 by first searchingwithin the SLB for the ESID. If the particular ESID is within the SLB (ahit), then the ESID indicates the mapped or corresponding VSID. If theESID is not within the SLB (a miss), then the operating system or othercomponent may search the main memory within the segment table 208B forthe ESID. If the ESID is within the segment table (a hit), then the ESIDindicates the mapped VSID. If the ESID is not within the SLB 208A orsegment table 208B (a miss), the ESID in some cases may be paged intomain memory from a storage device (e.g., disk) and the main memorysegment table (and SLB) may accordingly be updated.

If the ESID is found within the SLB 208A or segment table 208B, then thevirtual address 204 may be obtained. A virtual address may be anintermediate address that is generated by the processor or processorcore (or hypervisor) during the address translation process byconcatenating a VSID, a page index, and a byte offset. As discussedabove, the VSID may be obtained from the SLB 208A or segment table 208B.The page index of the virtual address 204 may be obtained from the pageindex of the effective address 202. Likewise, the byte offset of thevirtual address 204 may be obtained from the byte offset of theeffective address 202. The byte offset in FIG. 2 may indicate where alocation is within a particular page.

The VSID and page index, in some embodiments, comprise the virtual pagenumber (VPN). In order for the virtual address 204 to be translated tothe physical address 206, a hypervisor or other component may search fora particular VPN within the TLB 210A, and, if not found in the TLB 210A,the page table 210B. The TLB 210A and page table 210B may both includeat least two fields (not shown in FIG. 2). A first field may include theVPN and a second field may include a corresponding RPN. Accordingly, ifthe VPN of the virtual address 204 is located (a hit), then itscorresponding RPN may be located. In some embodiments, the hypervisor orother component may first search the TLB 210A for the VPN. If the VPN isnot within the TLB 210A (a miss), then the hypervisor or other componentmay search within the page table 210B. If the VPN is not within the pagetable 210B (miss), then a page fault may occur and the pagecorresponding with the VPN may be loaded into main memory (RAM) from ahard drive or non-volatile memory and the memory page table may beupdated with the VPN and corresponding RPN.

If the VPN is found within the TLB 210A or page table 210B, then thephysical address 206 may be obtained. A physical address may be a finaladdress that is generated by the processor or processor core (orhypervisor) during the address translation process. As discussed above,the VSID may be obtained from the SLB 208A or segment table 208B. Thepage index of the physical address 206 may be obtained from the pageindex of the virtual address 204. Likewise, the byte offset of thephysical address 206 may be obtained from the byte offset of the virtualaddress 204. After the physical address 206 is obtained, it may betransmitted to a memory subsystem and may be utilized to access memoryand devices (e.g., Peripheral Component Interconnect Express (PCIE)devices). A physical address may correspond to some physical resource inthe system, such as an address in Random Access Memory (RAM).

FIG. 3 is a flow diagram of an example process 300 for identifying staleentries within address translation cache, according to embodiments. Insome embodiments, the process 300 may begin at block 302 when anoperating system or other module (e.g., test case module 146 of FIG. 1)runs a first test case, which includes determining that a set of source(i.e. beginning, higher level, etc.) addresses are mapping to a firstset of addresses. A “test case” as described herein may refer to a setof variables or conditions under which a tester (e.g., an operatingsystem, a user, program module, etc.) may determine whether a systemunder the test satisfies requirements or works correctly. Test cases mayinclude such information as preconditions, a set of input values, a setof expected results, how to execute and check results, and/or expectedpost conditions, or any other suitable information. The test case inFIG. 3 may be utilized to test what particular set of addresses thesource addresses are mapping to. Test cases are described in more detailbelow. In some embodiments, the first set of addresses are virtualaddresses. Accordingly, an SLB or segment table may be utilized todetermine a virtual address, as described above. In some embodiments,the first set of addresses (and second set of addresses) are physicaladdresses (e.g., RPNs). Accordingly, a TLB or page table may be utilizedto determine a physical address, as described above. Likewise, the setof source addresses may either be effective addresses or virtualaddresses. The term “set of” as disclosed herein may mean one or moreof. The process 300 may be performed at any suitable time, such as toboot up, every time there is a context switch, at timed intervals,and/or when a new system is being tested prior to shipping, etc.

In an example illustration, the operating system may access or select 20effective addresses to begin the first test case. Address selection toperform test cases may be performed in various manners. For example, theoperating system may randomly select a particular quantity of addresses(e.g., addresses 10, 20, 50, etc.) until all the addresses are selectedfor testing. Alternatively, the operating system may select batchedaddresses in a particular order, e.g. consecutive. For example, thefirst 30 effective addresses may be selected for the first test case ata first time. At a second time, the next 30 effective addresses may beselected for another test case and so on until all of the addresses areselected for testing. In some embodiments, address selection may betime-based. For example, 20 addresses may be selected every 5 minutesuntil each address is tested.

Per block 304, the operating system and/or processor may determinewhether any of the first set of addresses are not within an addresstranslation cache table (e.g., TLB, SLB, etc.). Per block 306, if anyone of the first quantity of addresses are not within a translationcache table, then the operating system and/or processor may update theaddress translation cache table or memory table (i.e., pagetable/segment table), which may depend on the level of the miss. Forexample, if the set of source addresses was a virtual address and if thefirst set of addresses were not within TLB, but within a page table, anentry may be added to the TLB to reflect the mapping of the first set ofaddresses from a set of virtual source addresses to a set of physicaladdresses. However, if the first set of addresses were neither withinthe TLB or the page table, a fault may occur such that the address maybe paged in from disk and read only into the memory page table. In someembodiments, if any of the first set of addresses were never written toTLB (or SLB), then the processor, operating system, and or module (e.g.,test case module 146 of FIG. 1) may mark or flag the addresses such thatthese addresses will not be included in test cases because test casesmay identify stale entries within the address translation caches only.

Per block 308, the operating system (e.g., test case module 146) and/orprocessor may change the mapping specified in block 302 by mapping theset of source addresses to a second set of addresses within a tablestored in memory (e.g., page table or segment table). Changing the firstset of addresses to a second set of addresses for a set of data mayeffectively simulate a context switch. Accordingly, the address space(e.g., virtual or effective) of the first set of addresses switches orchanges address mappings, which causes the first set of addresses tochange to a second set of addresses during address translation for a setof data. For example, the first set of addresses may include a firstreal page that was previously mapped to by a first virtual address(source address) within a page table. The second set of addresses mayinclude a second real page that is now mapped to by the first virtualaddress within the page table. As discussed above, when switching fromone process to another, the hardware and/or operating system shouldensure that the about-to-be-run process does not accidently utilizeaddress translations from some previously run process. Block 308 maytherefore force one or more processors to initiate a flush operation inresponse to the changing of addresses. The changing of addresses isdescribed in more detail below. In some embodiments,

Per block 310, the operating system and/or the processor may execute orinitiate, in response to the changing a mapping, a flush operation (orseries of flush operations) to invalidate one or more entries within theaddress translation cache table. For example, when a first processorperforms an operation to flush a test address it's TLB, it may send aflush request to a second processors requesting that the secondprocessor flush the test address from the second processor's TLB. Theentries may correspond with each of the first set of addresses andtherefore be considered stale entries that are no longer valid foraddress translation. In some embodiments, the flush operation(s) mayinclude flushing an entire address translation cache. In otherembodiments, the flush operation may include flushing only those entrieswithin the address translation cache that are the stale entries (i.e.,the first set of addresses). In some embodiments, a single processor orprocessor core may initiate each of the flush operations associated witheach of the first quantity of addresses. In some embodiments, two ormore processors (or processor cores) may initiate each of the flushoperations associated with each of the first set of addressessubstantially simultaneously. For example, if the first set of addressesincluded address 1 (corresponding to TLB entry 1), address 2(corresponding to TLB entry 2), and address 3 (corresponding to TLBentry 3), a first processor (or core) may initiate a first flushoperation to flush TLB 1, a second processor (or core) may initiate asecond flush operation to flush TLB 2, and a third processor (or core)may initiate a third flush operation to flush TLB 3. In someembodiments, each of the flush operations may occur substantiallysimultaneously (e.g., in parallel) resulting in a race condition, asdescribed above, in order to determine whether there are any staleentries within address translation cache.

In some embodiments, the executing of the one or more flush operationsare in response to the processor or processor core issuing a tlbie(Translation Lookaside Buffer Invalidate Entry) instruction. The tlbieinstruction issued by a first processor or core to other processorsmakes a TLB entry invalid in the other processor's TLB for asubsequently requested effective-to-physical address translation or asubsequently requested effective-to-virtual-to-physical addresstranslation. In some embodiments, a TLB shootdown may also include anissuance of an inter-processor interrupt that causes the localinvalidation of a TLB entry. If a processor issues a tlbie instruction,other processors receive a message that indicates the issuance of thetlbie instruction. That message may also include a page size and a pagestarting address, corresponding to the page to be invalidated.

Per block 312, the operating system (e.g., test case module 146) and/orprocessor may run a second test case to test whether any of the set ofsource addresses are mapping to the first set of addresses (i.e., theinvalid addresses). The running of the second test case may includechecking a “results” section of the second test case, which specifieswhether any of the set of source addresses are mapping to the first setof addresses. Test cases are discussed in more detail below. Per block314, the operating system (e.g., test case module 146) and/or processormay determine whether any of the set of source addresses map to any ofthe respective first set of addresses (e.g., whether any virtual addressof the set of source addresses were mapping to a RPN specified in thefirst test case) in any of the respective SLBs and TLBS associated witheach of two or more processors. Per block 316, if none of the first setof addresses were accessed or mapped to during the second test case,then the operating system (e.g., test case module 146) and/or processormay determine that there are no stale entries within the addresstranslation cache (e.g., all of the entries were successfullyinvalidated). For example, if each of the set of source addresses in thesecond test case is mapped to the pages specified in the second testcase (i.e., the pages that were changed per block 308), then all of theentries within address translation cache may have been successfullyinvalidated and the mappings may have been derived from a correctlyupdated memory table (e.g., page table or segment table).

Per block 318, if any of the first set of addresses were accessed ormapped to during the second test case, then the operating system (e.g.,test case module 146) and/or processor may determine that there is atleast one stale entry within at least one of the address translationcache tables. The determination at block 318 may be an inference thatthere are stale entries because the operating system and/or processor inblock 308 changed the real (or virtual) addresses of each of the firstset of addresses for address translation within a table stored in memory(e.g., page table). Accordingly, any of the set of source addressesshould have mapped to the changed addresses (second set of addresses)instead of the first set of addresses not only within a page table orsegment table, but also within the address translation caches. Theoperating system (e.g., test case module 146) and/or processor mayidentify whether there are any stale entries by comparing the first testcase with the second test case. For example, a tester (e.g., test casemodule 146 or user) may run the first test case and see that a sourceaddress maps to a first physical address. However, if the tester runs asubsequent second test case and sees that the source address still mapsto the first physical address in one or more caches (even aftersimulating a context change in block 308), then the tester may identifythe first physical address as being the address that contains acorresponding stale entry in an address translation cache. Therefore,any address that mapped to a page that was specified in the first testcase may mean that the processor or core associated with the cache didnot receive or process the flush operation for the first address inorder to clear its address translation cache notwithstanding that anactual change was made in an associated page table structure.

FIG. 4A is a diagram of an example page table entry illustrating how aphysical address may be changed (e.g., block 308 of FIG. 3), accordingto embodiments. FIG. 4A includes a page table entry 402A, which includesa Virtual Page Number (VPN), additional bits 404 (e.g., protection bit,valid bits, dirty bits, referenced bits, etc.), and a Physical PageNumber (RPN). FIG. 4A further includes the page table entry 402B, whichmay be the page table entry 402A after page table entry 402A hasreceived an address change. Page table entry 402B may include a VirtualPage Number (VPN), additional bits, and a Physical Page Number (RPN),which includes an offset in order to make the RPN a larger size. Theoperating system or other component may provide the offset by maskingand specifying what particular set of bits need to be set, which maydetermine how many bits are being added to the RPN for page table entry402B. In some embodiments, the operating system or other component mayadd the offset within the RPN (e.g., zero out a defined set of bitswithin RPN) such that the RPN is smaller for page table entry 402B whencompared to the RPN of page table entry 402A. As discussed above, theoperating system or other component may change the RPN in order toeffectively change a context, which may require one or more TLB flushoperations to occur. Accordingly, page table entry 402A may be the pagetable entry the first test case utilizes. Page table entry 402B may bethe page table entry the second test case test utilizes in order toidentify stale entries within TLB, as discussed above.

In an example illustration, the RPN of page table entry 402A may be3B9AC00h, which may be 1 GB. In order to change the mapping from the VPNof page table entry 402A to the RPN of page table entry 402B to make theRPN larger, the operating system may add a known offset such as 3B9AC00h(e.g., 1 GB), which now makes the RPN of table entry 402B 77359400h (2GB) instead of 3B9AC00h (e.g., the RPN of table entry 402A).Accordingly, when the RPN has changed within the page table entry, theVPN will now point to a different RPN. In a page table full of severalentries, the operating system may increment each RPN table entry withthe same offset (i.e., a fixed offset). For example, using theillustration above, another page table entry RPN may have originallybeen 3B9AC00h, which may be 1 GB, plus an addition of 3E800h, which maybe 256 k. With the same offset of 3B9AC00h (e.g., 1 GB) added to eachentry, the RPN may be 77359400h plus 3E800h or 3B9EB200h (e.g., 2 GBplus 256 k).

FIG. 4B is a diagram of an example segment table entry illustrating howa virtual address may be changed (e.g., block 308 of FIG. 3), accordingto embodiments. The same or analogous principle illustrated in FIG. 4Ato change pages may also be applied to intermediate page levels (e.g.,virtual pages). FIG. 4B includes segment table entry 406A, which mayinclude an Effective Segment ID (ESID), additional bits 408, and aVirtual Segment ID (VSID). FIG. 4B may also include segment table entry406B, which may be the segment table entry 406A after an address changehas been made. Segment table entry 406B may also include a VPN,additional bits, and am ESID and VSID, which includes an offset in orderto make the VSID a larger size. The operating system or other componentmay provide the offset by masking and specifying what particular set ofbits need to be set, which may determine how many bits are being addedto the VSID for segment table entry 406B. In some embodiments, theoperating system or other component may add the offset within the VSID(e.g., zero out a defined set of bits within VSID) such that the VSID issmaller for segment table entry 406B when compared to the VSID ofsegment table entry 406A. As discussed above, the operating system orother component may change the VSID in order to effectively change acontext, which may require one or more SLB flush operations to occur.Accordingly, segment table entry 406A may be the segment table entry thefirst test case utilizes. Segment table entry 406B may be the segmenttable entry the second test case test utilizes in order to identifystale entries within SLB, as discussed above.

The page table entries 402A and/or 402B may illustrate a simplistic viewof such entries, and may include more or less bit fields with differentpage table configurations than illustrated. For example, the page tableto which to page table entries 402A and/or 402B belong to may beinverted page tables, multilevel page tables, virtualized page tables,nested page tables, or any other type of page table.

FIG. 5 is a diagram of an example of two test cases, according toembodiments. The table 500 illustrates two test cases, test case numbers1A and 1B. In some embodiments the test cases 1A and 1B may test whetherthere are stale entries within a TLB, as illustrated in FIG. 5. In otherembodiments however, the test cases 1A and 1B may test whether there arestale entries within SLB. The two test cases may include various fields(i.e., columns) such as: a “test case ID” field, a “description” field,a “preconditions” field, a “steps” field, an “expected results” field,and an “actual results” field. The table 500 and test cases may includemore or less fields than illustrated in FIG. 5. For example, the tablemay include additional fields such as a “pass/fail” field, a “postconditions” field, a “status” field, etc. Although each of the testcases 1A and 1B illustrates textual prose descriptions within eachfield, in some embodiments, the descriptions in each field may bedescribed by symbols, numbers, and/or other shorthand representations.

The “description” field may indicate a goal of the test case, which fortest case 1A may be to identify a first set of RPNs for a first set ofvirtual (source) addresses. The “preconditions” field may indicate anyconditions or requirements that must occur before the test case isinitiated or completed. For test case 1A, the precondition may be thatthe computing system should be in virtual memory mode such that addresstranslation will be utilized. The “steps” field may indicate the stepsrequired to complete a test case. For test case 1A, the first step maybe to access a first set of virtual addresses. In order to access thefirst set of virtual addresses, an effective address is translated(e.g., via a SLB or segment table) to the virtual address. The secondstep may include identifying the set of RPNs for each of the first setof virtual addresses. Specifically, when a virtual address is translatedto a physical address, the VPN of the virtual address will map (e.g.,via the TLB or page table) to a particular RPN. Accordingly, the secondstep may be to identify each of the address space mappings of virtualaddresses to the RPNs of the physical addresses. The “expected results”field may indicate what the results should be, and according to testcase 1A, the expected results are that each of the first set of virtualaddresses should map to the first set of RPNS within a page table andwithin a TLB. The “actual results” (i.e., observed results) may be thateach of the first set of virtual addresses mapped to the first set ofRPNS in each of the page tables and address translation caches.

For test case 1B, the “description” field may be to identify a secondset of RPNs for the same first set of virtual (source) addresses. Theremay be at least 2 “preconditions” for test case 1B. The firstpreconditions may be that the first set of RPNS for the first set ofvirtual addresses (accessed in test case 1A) should have been changed(or the appropriate commands should have been issued to change amapping) to a second set of RPNS in a page table. Accordingly, the firstprecondition may specify that a mapping change (virtual to physicaladdresses) occurred for the first set of virtual addresses such that thephysical addresses are now different within the page table. In someembodiments, when the physical addresses are changed within a page table(or segment table), then the same change is made within a TLB (or SLB).In an example illustration, referring back to FIG. 4A, the first set ofRPNs may correspond to the RPN of page table entry 402A and the secondset of RPNs may correspond to the RPN of page table entry 402B, whichhas received an offset to change the RPN (which changes the physicaladdress or mapping from the source address). In some embodiments, thefirst precondition in the second test case 1B (i.e., the changing amapping) is a part of the second test case 1B instead of a precondition.The second precondition may be that a TLB flush command has been issuedfor each entry associated with each of the first set of RPNs (i.e., theRPNs that were mapped to during the first test case) in each of theaddress translation caches. As discussed above, the TLB flush operationsmay be executed simultaneously and because a race condition may occur,stale TLB entries may still be contained in a TLB even though addressmappings have changed within a page table. In some embodiments thesecond precondition for test case 1B (i.e., the TLB flush operation) isa part of the test case 1B instead of a precondition to test case 1B.

The first step for test case 1B may be to access the first set ofvirtual addresses (from test case 1A). The second step may be toidentify the set of RPNs for each of the first set of virtual addresses.Accordingly, the RPNs that each of the first set of virtual addresseshave mapped to may be observed (e.g., within a TLB). For the expectedresults, it is expected that none of the first set of virtual addressesshould map to the first set of RPNs, but instead each of the first setof virtual addresses should map to the second set of RPNs. This may bethe expected result because of the precondition of changing the mappingof each of the first set of virtual addresses from the first set of RPNsto the second set of RPNs within a page table. However, given the secondprecondition of the TLB flush operation, if there are stale entrieswithin one TLB or two or more TLBs, then the associated first set ofvirtual addresses will map to the corresponding first set of RPNs,instead of the corresponding second set of RPNs (e.g., within a TLB).This may occur because a system may first look into address translationcache, as opposed to a table in memory (e.g., page table), and if anentry was not flushed, the invalid mapping will be utilized within theaddress translation cache instead of the valid mapping stored in memory(i.e., the mapping changed as part of precondition 1 of test case 1B).For example, an invalid TLB entry may have been inadvertently retainedsuch that a virtual address is being translated to the old physicaladdress (which includes the first set of RPNs) contained in a TLBinstead of the new physical address (which includes the second set ofRPNs) contained in a page table. According to the “actual results” ofthe test case 1B, one address of the first set of virtual addresses,e.g., address X, may have been mapped to a physical address in the firstset of RPNs. Therefore, virtual address X may be associated with anentry within a TLB that is stale or invalid.

In some embodiments, the table 500 and the test cases 1A and 1B mayrepresent test cases to test whether there are stale entries within aSLB, as opposed to a TLB. Accordingly, for example, for test case 1A,the description may be to identify a first set of VSIDs (or virtualaddresses) for a first set of ESIDs in one or more segment tables. Thesteps should include, first, accessing a first set of effectiveaddresses (or ESIDs). The second step may include identifying the set ofVSIDs for each of the first set of ESIDs. The expected results may bethat each of the first set of ESIDs should map to the first set ofVSIDs. The actual results may be that each of the first set of ESIDs maymap to the first set of VSIDs.

For the second test case 1B, the description may be to identify a secondset of VSIDs for the same (first) set of ESIDs. The first preconditionof test case 1B may be that the first set of VSIDs within the segmenttable for the first set of ESIDs should be have been changed to a secondset of VSIDs within a segment table in memory. The firs precondition maybe an actual part of the test case 1B. The second precondition may bethat an SLB flush operation commands should have been issued for each ofthe first (or second) VSIDs in one or more SLBs. The second preconditionmay be an actual part of the test case 1B. The steps for test case 1Bshould be first to access the first set of ESIDs (e.g., as illustratedin FIG. 2). The second step may be to identify the set of VSIDs for eachof the first set of ESIDs. The expected results may be that none of theESIDs in the first set of ESIDs should map to the first set of VSIDs(because of the change in the segment table as specified in theprecondition), but the first set of ESIDs should map to the second setof VSIDs. The actual results may be that one of the first set of ESIDsmapped to one of the first set of VSIDs in one or more of the SLBs,which may mean that there may be one or more stale entries within a SLB.

Aspects of the present invention may be a system, a method, and/or acomputer program product. The computer program product may include acomputer readable storage medium (or media) having computer readableprogram instructions thereon for causing a processor to carry outaspects of the various embodiments.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofembodiments of the present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of embodiments of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

What is claimed is:
 1. A system for identifying stale entries within anaddress translation cache, the system comprising: a computing devicehaving a processor; and a computer readable storage medium havingprogram instructions embodied therewith, the program instructionsreadable/executable by the processor to cause the system to perform amethod, the method comprising: running, by the computing device, a firsttest case, the first test case to determine which particular set ofphysical addresses a first set of virtual addresses are mapping to, therunning of the first test case including determining that the first setof virtual addresses are mapped to a first set of physical addresses;changing, by the computing device, a mapping in a page table stored inmemory, the page table mapping the first set of virtual addresses, for aset of data, to the first set of physical addresses, the changing of themapping including mapping the first set of virtual addresses to a secondset of physical addresses, wherein the changing a mapping includesadding an offset quantity of bits to the first set of physicaladdresses; executing, by the computing device and in response to thechanging of the mapping, one or more flush operations to invalidate oneor more entries within one or more Translation Lookaside Buffer (TLB)address translation caches, the one or more entries corresponding to thefirst set of physical addresses, the TLB address translation cachesbeing buffers within a processor that maps virtual addresses to physicaladdresses for address translation; running, by the computing device andin response to the executing of the one or more flush operations, asecond test case, the second test case to test whether any of the firstset of virtual addresses are mapping to the first set of physicaladdresses; determining, by the computing device and based on the secondtest case, that at least one of the first set of virtual addresses aremapping to at least one of the first set of physical addresses;determining, by the computing device and based on comparing the firsttest case with the second test case, that at least one of the one ormore entries were not invalidated by the one or more flush operations;running, by the computing device, a third test case, the third test caseto determine which particular set of virtual addresses a first set ofeffective addresses are mapping to, the running of the third test caseincluding determining that the first set of effective addresses aremapped to a second set of virtual addresses; changing, by the computingdevice, a mapping in a segment table stored in the memory, the segmenttable mapping the first set of effective addresses, for a second set ofdata, to the second set of virtual addresses, the changing of themapping including mapping the first set of effective addresses to athird set of virtual addresses, wherein the changing a mapping in thesegment table includes adding an offset quantity of bits to the secondset of virtual addresses; executing, by the computing device and inresponse to the changing of the mapping in a segment table, one or moreflush operations to invalidate one or more entries within one or moreSegment Lookaside Buffer (SLB) address translation caches, the one ormore entries within the one or more SLB address translation cachescorresponding to the second set of virtual addresses, the SLB addresstranslation caches being buffers within the processor that mapseffective addresses to virtual addresses for address translation;running, by the computing device and in response to the executing of theone or more flush operations to invalidate one or more entries withinone or more SLB address translation caches, a fourth test case, thefourth test case to test whether any of the first set of effectiveaddresses are mapping to the second set of virtual addresses;determining, by the computing device and based on the fourth test case,that at least one of the first set of effective addresses are mapping toat least one of the second set of virtual addresses; determining, by thecomputing device and based on comparing the third test case with thefourth test case, that at least one of the one or more entries of withinthe one or more SLB address translation caches were not invalidated; andwherein the first test case, the second test case, the third test case,and the fourth test case each share a plurality of fields that aredisplayed by the computing device, the plurality of fields including: afirst field identifying a description of each of the test cases, asecond field identifying one or more preconditions needed in order tostart each of the test cases, a third field identifying one or moresteps needed to complete each of the test cases, a fourth fieldidentifying one or more expected results for each of the test cases, anda fifth field identifying one or more actual results for each of thetest cases.